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APPELLANTS' REPLY BRIEF UNDER 37 CFR 1.193 (b) 

This Reply Brief is being filed in response to new points of argument raised in 
the Examiner's Answer mailed December 1 1 , 2006. Appellants respond to these new 
points of argument as follows. 
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STATUS OF CLAIMS 

Claims 1-18 are pending in this application. 

Claims 1-18 stand finally rejected and are on Appeal. 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



Whether the rejection of claims 1-18 are patentable over the cited references, 
with regards to 35 U.S.C. 103(a) is in error. 
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ARGUMENTS 



A. The rejection of claims 1 and 12 as unpatentable over Bresse et al. in view 
of Mizokawa is in error. 
1 . Emotions are not Modalities 

Claim 1 is limited to "determining the preferred modalities of the individual." In 

the Final Office Action and in the Examiner's Answer, the Examiner admits Breese 

does not disclose preferred modalities, but states "Mizokawa teaches a system which 

including 'determining the preferred modalities of the individual' [col 6, line 55-60], 

preferred modalities of the individual corresponds to Mizokawa's user's emotions as 

detailed in col 6, line 58-60." Page 4 of the Final Office Action and Page 4 of the 

Examiner's Answer. The col 6, line 55-60 passage is as follows: 

Further, based on the presumed emotions, the user's emotion recognition 
unit A2 determines the user's current emotion. In the above, the several 
patterns of emotions are obtained by categorizing in advance the emotion 
expressions of the user into several patterns of emotions such as joyful, 
sad, surprised, angry, disgusted, and fearful. 

As detailed in the filed application, "humans have as many modalities as they 
have senses. For the purposes of a simplified discussion, we use a simplified web 
browser interface for reference. We can match the sensory modalities to the 
presentation via a grid. The grid [shown in the application] represents three of the 
many sensory modalities most humans share; visual, auditory and kenisthetic." 
Paragraph 54 of the application. This statement should not be read to limit the 
modalities to the five common senses. The "sensory" modalities are "input" modalities 
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and those having ordinary skill in the art recognize that sensory modalities include 
things like visual, auditory, kinesthetic, olfactory, gustatory, vistubulary, proprioceptive, 
tactile and vomeronasal. There is some conjecture if the PNI is another sensory 
modality. Consciousness researchers are discovering we have more senses than we 
previously realized (vomeronasal, for example, wasn't recognized as a sense until the 
mid-1990s). It is sufficient to say that we have more sensory modalities than the five a 
school child can name and it is reasonable to believe in the future we may understand 
we have more senses than we recognize today. 

As stated in the Appeal Brief, the Applicant does not understand how the 
Examiner determines that modalities encompass emotions. On pages 9-1 1 of the 
Examiner's Answer, the Examiner addresses this matter. Specifically, the Examiner 
does not disagree with the Applicant's definition of modalities as it used in the art and 
defined in the application. Rather, the Examiner states "as recognized from the 
Mizbkawa's reference that characteristics of sounds, voices, ... temperature, 
brightness, ... are directly related to user's emotions ... therefore, emotions are 
directly related to modalities." Basically, the Examiner's argument appears to be that 
the Mizokawa computer senses input in a manner comparable to a person's sensory 
modalities, and Mizokawa determines emotions, therefore recognizing emotions is 
determining preferred modalities. This logic is clearly flawed. Defining whether an 
individual is sad or joyful is not comparable to determining if an individual has a 
preference toward the auditory or the visual. Emotions are not modalities. 

The Examiner's comments on page 1 1 of the Examiner's Answer better isolates 
the requirements of the claims and displays the difference between the teachings of 
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Mizokawa and the claimed invention. As the Examiner cites from paragraph 66 of the 
original filed application, "when the grid movement is determined, the modality 
summations can be selected to determine if the viewer's attention is focused on visual, 
auditory, kinesthetic, or other related cues." This statement from the application 
encapsulates "determining preferred modalities of an individual." Determining 
which modality the individual is focused on determines which modality the individual 
prefers. On page 1 1 of the Examiner's Answer, the Examiner states "In response to 
Applicant's argument that the references fail to show certain features of Applicant's 
invention, it is noted that the features upon which the Applicant relies ... are not recited 



None of the references cited disclose determining preferred modalities of an 
individual. 



2. 

Claim 1 and claim 12 reference "preferred modalities." The Examiner makes 
conclusory statements that emotions are modalities. To the extent these statements 
are given any weight, the Examiner has ignored the term "preferred" in that he has not 
suggested Mizokawa discloses how to determine a preferred emotion of an individual. 
"All words in a claim must be considered in judging the patentability of that claim 
against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 
1970). One would assume everyone would want to be joyful and this exercise would 
be fruitless, but it is worth noting that Mizokawa does not disclose how to determine 
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what emotion is preferred by an individual and the Examiner does not suggest 
Mizokawa discloses how to determine which emotion is preferred by an individual. 



3. Abstract Idea 

The present file wrapper includes two non-final office actions, two final office 

actions and an advisory action. Not a single claim has been amended since the 

application was filed. Yet, the Examiner waited until after a second appeal brief was 

filed in this prosecution to raise the notion of an "abstract idea" rejection. This matter 

is not ripe for argument as it raised for the first time in an Examiner's Answer and it is 

improperly raised in an appeal directed solely to 103 rejections. That said, the 

following citation from paragraphs 66 and 67 of the application provides again the 

definition for "preferred modalities" and the reason determining/calculating preferred 

modalities provides a useful, tangible, and concrete result: 

When grid movement is determined, the modality summations can 
be selected to determine if the viewer's attention is focused on visual, 
auditory, kinesthetic or other related cues. Based on the results of this 
equation, the web server can prepare in real time what the next 
presentation and interface should be in order to capture more of the 
viewer's attention, by presenting the web content in modalities which the 
viewer has nonconsciously selected. 

(emphasis added) 



4. Calculating Preferred Modalities 

On page 13 and 14 of the Examiner's Answer, the Examiner makes a separate 
argument of "calculating preferred modalities," apparently feeling it needed to be 
argued separately from "determining preferred modalities." The Applicant is not 
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raising a separate argument here and believes the limitation "calculating preferred 
modalities" is novel over the cited art for the same reasons "determining preferred 
modalities" is novel over the prior art. 



As can readily be seen, the Applicant's claim is limited to 
determining/calculating preferred modalities, rendering the claims allowable in view of 
Breese and Mizokawa for at least this reason. Therefore, the Applicant respectfully 
submits claims 1 and 12 are allowable in that they recite features and steps that are 
neither disclosed, taught, nor suggested by Breese in view of Mizokawa. Furthermore, 
the Applicant submits that claims 2-1 1 and 13-18 are allowable for at least the reason 
that they depend either directly or indirectly from allowable independent claims. In re 
Fine , 5 U.S.P.Q. 2d 1596, 1608 (Fed. Cir. 1988). 

1 . Modifying the Environment 

Claims 2, 3, 7, 8, 13, 16, and 17 are directed toward modifying the environment 
(or environmental unit or modifiable environmental unit) in response, in varying ways, 
to the sensed information and/or the determined/calculated preferred modalities. In 
each of these cases, the Examiner responds (as shown on pages 15 and 17of 
Examiner's Answer) that the computer, be it Mizokawa or Breese, records changing 
emotions. 

Paragraph 13 of the originally filed application states: 
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The premise of the present invention is to have sensing devices, 
many of which already exist in an individual's environment or which could 
be inconspicuously arranged so as not to alter the environment, sensing 
an individual's actions. A processing unit then interprets those actions, 
analyzing the psycho- and cognitive-motor behavior and other activity to 
identify preferred modalities of the individual. Modifiable environmental 
units are then modified to make the environment more consistent with the 
individual's preferred modalities. 

Tracking data is not the same, or remotely similar, to changing the environment of an 

individual. It is unclear how the Examiner is equating changes in sensed emotions to 

changing an individual's environment. This question has been asked in no fewer than 

five office action responses, two appeal briefs, and an Examiner Interview without 

additional clarity being provided. 

2. Linear Algebraic Transforms 

There is a difference between linear algebraic transforms and algebraic 
transforms. On page 18 of the Examiner's Amendment, the Examiner states Breese 
discloses algebraic transforms. The Examiner does not state Breese discloses linear 
algebraic transforms, which is a limitation of claim 5. Breese does not disclose linear 
algebraic transforms. Thus, the cited references fail to disclose all the limitations of 
claim 5. This comment has been included in the office action response filed in 
October of 2005 and in the Appeal Brief, but the Examiner has never addressed the 
difference between linear algebraic transforms and algebraic transforms. 

3. Order Preferred Combination of Modalities 
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Claims 6, 8, and 9 are limited to ordering a preferred combination of modalities. 

In the rejection of claim 1, the Examiner rejects preferred modalities based on 

Mizokawa's teaching of emotion recognition. Therefore, a rejection of this limitation 

logically following the rejection of claim 1 should involve an argument that a reference 

teaches a preferred combination of emotions. The Examiner cites to the following 

passage to argue Mizokawa discloses ordering a preferred combination of modalities 

(col. 7, lines 1 1-24 of Mizokawa): 

The user's command recognition unit A3 recognizes and establishes the 
user's command information based on indirect command information 
obtained by analyzing the meaning of the sound/voice recognized by the 
auditory input device and by analyzing information on the user's gestures 
recognized by the visual input device, and based on direct command 
information inputted via the keyboard. The command information includes 
not only direct commands such as "search" or "lay out an action plan" but 
also a degree of requirement for the command such as "urgent" and 
"normal". The degree of requirement can be recognized by, for example, 
the loudness of the user's voice or the speed of the user's speech inputted 
via the auditory input device, or the user's gesture inputted via the visual 
input device. 

The Mizokawa passage discloses an ordering of tasks based on the emotions 
interpreted from the user. The reference, as previously discussed, does not teach 
modalities. Further, the reference does not teach combinations of or ordering of 
modalities, or the Examiner's interpretation of modalities. 



4. Claim 9 

Claims 9 is limited to "wherein the combination and order of modalities is 
calculated by an equation: ^((IGKSxi/SitMZGrtdx/dtiNfclGidti * >F(G)." The 
Examiner rejects this claim as obvious over Breese in view of Mizokawa despite 
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stating on page 19 of the Examiner's Answer, "It is noted that both Breese, Mizokawa 
do not show modalities calculation by an equation." (Emphasis added). Claim 9 
is not limited to using "an equation." Claim 9 is limited to a very specific equation, 
which is not disclosed in any reference cited by the Examiner. As the Examiner 
admits, the references fail to disclose every limitation of the claimed invention, and 
thus should be allowed. To establish prima facie obviousness of a claimed invention, 
all the claim limitations must be taught or suggested by the prior art. In re Royka, 490 
F.2d 981 , 1 80 USPQ 580 (CCPA 1974). It is well established at law that, for a proper 
rejection of a claim under 35 U.S.C. §103 as being obvious based upon a combination 
of references, the cited combination of references must teach, disclose, or suggest, 
either implicitly or explicitly, all elements/features/steps of the claim at issue. See, 
e.g.. In re Dow Chemical , 5 U.S.P.Q. 2d 1529, 1531 (Fed. Cir. 1988), and In re Keller , 
208 U.S.P.Q. 871, 881 (C.C.P.A. 1981). 



In view of the foregoing, it is respectfully requested that the Examiner's rejection 
of the subject application be reversed in all respects. 



CONCLUSION 



Respectfully submitted, 
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